Automatic Change Analysis in Satellite Images Using Binary Descriptors and Lloyd–Max Quantization

In this letter, we present a novel technique for unsupervised change analysis that leads to a method of ranking the changes that occur between two satellite images acquired at different moments of time. The proposed change analysis is based on binary descriptors and uses the Hamming distance as a similarity metric. In order to render a completely unsupervised solution, the obtained distances are further classified using vector quantization methods (i.e., Lloyd's algorithm for optimal quantization). The ultimate goal in the change analysis chain is to build change intensity maps that provide an overview of the severeness of changes in the area under analysis. In addition, the proposed analysis technique can be easily adapted for change detection by selecting only two levels for quantization. This discriminative method (i.e., between changed/unchanged zones) is compared with other previously developed techniques that use principal component analysis or Bayes theory as starting points for their analysis. The experiments are carried on Landsat images at a 30-m spatial resolution, covering an area of approximately 59×51 km2 over the surroundings of Bucharest, Romania, and containing multispectral information

Emotion Recognition System from Speech and Visual Information based on Convolutional Neural Networks

Emotion recognition has become an important field of research in the human-computer interactions domain. The latest advancements in the field show that combining visual with audio information lead to better results if compared to the case of using a single source of information separately. From a visual point of view, a human emotion can be recognized by analyzing the facial expression of the person. More precisely, the human emotion can be described through a combination of several Facial Action Units. In this paper, we propose a system that is able to recognize emotions with a high accuracy rate and in real time, based on deep Convolutional Neural Networks. In order to increase the accuracy of the recognition system, we analyze also the speech data and fuse the information coming from both sources, i.e., visual and audio. Experimental results show the effectiveness of the proposed scheme for emotion recognition and the importance of combining visual with audio data

A L2-Norm Regularized Pseudo-Code for Change Analysis in Satellite Image Time Series

The continuous progress in the acquisition of high-dimensional information (e.g., satellite image time series, or medical screening) im-plies an efficient characterization of changes that occur in a temporal series of data. A pseudo-encoding technique can be designed to represent the changes between two consecutive moments of time, based on the min-imization of a convex error function which has an analytical solution. The domain transformed feature vectors are grouped into clusters using K-Means. The proposed approach results in a better separation between classes and, thus, in an enhanced characterization of temporal changes. The experiments are done on 5 Landsat multispectral images at 30 me-ters spatial resolution, covering an area of approximately 59 X 51 km2 around Bucharest, Romania

Multimodal Satellite Image Time Series Analysis Using GAN-Based Domain Translation and Matrix Profile

The technological development of the remote sensing domain led to the acquisition of satellite image time series (SITS) for Earth Observation (EO) by a variety of sensors. The variability in terms of the characteristics of the satellite sensors requires the existence of algorithms that allow the integration of multiple modalities and the identification of anomalous spatio-temporal evolutions caused by natural hazards. The unsupervised analysis of multimodal SITS proposed in this paper follows a two-step methodology: (i) inter-modality translation and (ii) the identification of anomalies in a change-detection framework. Inter-modality translation is achieved by means of a Generative Adversarial Network (GAN) architecture, whereas, for the identification of anomalies caused by natural hazards, we adapt the task to a similarity search in SITS. In this regard, we provide an extension of the matrix profile concept, which represents an answer to identifying differences and to discovering novelties in time series. Furthermore, the proposed inter-modality translation allows the usage of standard unsupervised clustering approaches (e.g., K-means using the Dynamic Time Warping measure) for mono-modal SITS analysis. The effectiveness of the proposed methodology is shown in two use-case scenarios, namely flooding and landslide events, for which a joint acquisition of Sentinel-1 and Sentinel-2 images is performed

Spatio-Temporal Characterization in Satellite Image Time Series

Last years have witnessed an increased interest in the analysis of evolution of spatio-temporal structures in large data volumes. The current satellite remote sensing missions allow the recording of long Satellite Image Time Series (SITS) with passive and active sensors. However, the spatio-temporal characteristics of SITS imply different approaches. This paper aims at presenting an overview of these issues, and also some recent results based on Gibbs Markov Random Fields models that are used to describe the spatio-temporal patterns. In addition, in order to obtain an automatic analysis of these patterns, the problem of determining the optimal number of spatio-temporal clusters is also discussed. The experiments are carried on Landsat 7 multi-temporal and multi-spectral images and on Envisat ASAR images, both at 30 meters spatial resolution

SAR Imagery from the Perspective of Multiscale Chirplet Transform

In SAR (Synthetic Aperture Radar) imagery, a major concern is related to finding a way of classifying the objects retrieved in the data acquisition made by a sensor. Dealing with complex-valued data requires finding an adequate descriptor that will provide enough information about the object captured and, in the end, will determine the type of the object. Taking into consideration the fact that the data acquisition model for SAR is based on chirp signals, an immediate choice for getting the desired features is the Chirplet Transform, which, practically, transforms the time space into a frequency-chirprate space. Therefore, the main purpose of the present paper consists in defining the multi-scale 2-D Chirplet Transform and finding a way of characterizing objects captured in a SAR image by using the method mentioned

Retrieval of Similar Evolution Patterns from Satellite Image Time Series

Technological evolution in the remote sensing domain has allowed the acquisition of large archives of satellite image time series (SITS) for Earth Observation. In this context, the need to interpret Earth Observation image time series is continuously increasing and the extraction of information from these archives has become difficult without adequate tools. In this paper, we propose a fast and effective two-step technique for the retrieval of spatio-temporal patterns that are similar to a given query. The method is based on a query-by-example procedure whose inputs are evolution patterns provided by the end-user and outputs are other similar spatio-temporal patterns. The comparison between the temporal sequences and the queries is performed using the Dynamic Time Warping alignment method, whereas the separation between similar and non-similar patterns is determined via Expectation-Maximization. The experiments, which are assessed on both short and long SITS, prove the effectiveness of the proposed SITS retrieval method for different application scenarios. For the short SITS, we considered two application scenarios, namely the construction of two accumulation lakes and flooding caused by heavy rain. For the long SITS, we used a database formed of 88 Landsat images, and we showed that the proposed method is able to retrieve similar patterns of land cover and land use

Bag-of-Visual Words and Error-Correcting Output Codes for Multilabel Classification of Remote Sensing Images

This paper presents a novel framework for multilabel classification of remote sensing images using Error-Correcting Output Codes (ECOC). Starting with a set of primary class labels, the proposed framework consists in transforming the multiclass problem into binary learning subproblems. The distributed output representations of these binary learners are then transformed into primary class labels. In order to obtain robustness with respect to scale, rotation and image content, a Bag-of-Visual Words (BOVW) model based on Scale Invariant Feature Transform (SIFT) descriptors is used for feature extraction. BOVW assumes an a-priori unsupervised learning of a dictionary of visual words over the training set. Experiments are performed on GeoEye-1 images and the results show the effectiveness of the proposed approach towards multilabel classification, if compared to other methods